Learning to grasp using visual information

A scheme for learning to grasp objects using visual information is presented. A system is considered that coordinates a parallel-jaw gripper (hand) and a camera (eye). Given an object, and considering its geometry, the system chooses grasping points, and performs the grasp. The system learns while performing grasping trials. For each grasp we store location parameters that code the locations of the grasping points, quality parameters that are relevant features for the assessment of grasp quality, and the grade. We learn two separate subproblems: (1) to choose grasping points, and (2) to predict the quality of a given grasp. The location parameters are used to locate grasping points on new target objects. We consider a function from the quality parameters to the grade, learn the function from examples, and later use it to estimate grasp quality. In this way grasp quality for novel situations can be generalized and estimated. An experimental setup using an AdeptOne manipulator to test this scheme was developed. Given an object, the system takes one image of it with a stationary top-view camera, uses the image to choose two grasping points on the boundary, performs a grasping trial with a parallel-jaw gripper, and assigns a grade to the trial using an additional side-mounted camera. The system has demonstrated an ability to grasp a relatively wide variety of objects, and its performance improves with experience appreciably after a small number of trials.